Evaluation Question and Answer Analysis Report

The Evaluation Question and Answer Analysis report helps you analyze the quality of the questions and answers in an evaluation form. Using this report, you can:

  • Determine whether important questions are impacting the quality score as intended.

  • See if the evaluation form could be revised to align behaviors with overall performance.

You must have these permissions:

  • Dashboard > Dashboard Templates > Evaluation Question and Answer Analysis Report: On

  • Dashboard > Dashboards: View

  • Dashboard > Dashboards: Edit (optional)

If you cannot access the reporting templates or Dashboard, or if widget data is not loading, check with your administrator. The administrator can verify your role's reporting, dashboard, and widget permissions by visiting Admin > Security Settings > Roles and Permissions.

John Middleton is a quality manager at Classics, Inc. He wants to check on the questions and answers in the evaluation form he made, called Q1 Assessment. He opens the Evaluation Question and Answer Analysis report, defines a time period, and selects the team assigned to the evaluation. He then selects Q1 Assessment in the Form field. He notices question #3 in Q1 Assessment has a low Answer Count. Because question #3 is a free response question, this means not many agents answered the question. He revised question #3 to be clearer.

Data in this Report

The Evaluation Question and Answer Analysis report has two widgets – Question and Answer Analysis widget and Evaluation Question Details. Each widget has multiple columns that present data attributes and metrics. The table below provides a description for each column.

Question and Answer Summary Table

The Question and Answer Summary table has multiple columns that present data attributes and metrics. The table below describes each column. The data in the table can be filtered using the column-level filters available in the table.

Column Description
Section Section of the evaluation.
Question

Questions in the section. Questions marked with an asterisk * are required or critical.
Answer

Answers given for each question.

For multiple choice questions, each answer is given its own row. If the evaluators selected more than one answer, the selected answers are shown separated by a comma.

For free response questions that allow the evaluators to enter their own answer, the answer type will be displayed. The possible answer types are:

  • Text answer is given / is not given: A short, single line answer.

  • Text area answer is given / is not given: A long, possibly multiple line answer.

  • Hyperlink answer is given / is not given: An answer with a hyperlink.

  • Datetime answer is given / is not given: An answer with a date and time.

If the answer is edited in the evaluation, it is updated in the report.

Click the answer type to see details of each value that was entered, along with the number of times each answer was given.
Answer Count Number of times the answer was selected. For free response questions, this is the total count of answers.
Answer Representation %

Percentage of evaluators who chose the answer. For example, if 10 evaluators answered the question, and 4 selected the answer Yes, then the Answer Representation % for Yes is 40%.

For free response questions, the Answer Representation % is the percentage of evaluators who entered each answer type. For example, if 10 evaluators took the evaluation, and 7 answered the question with a date and time, then the Answer Representation % for Datetime answer is given is 70%, and the value for Datetime answer is not given is 30%.
Average Eval Score % Average of the evaluation scores for the agents who selected the answer. For free response questions, this value is always N/A.

Evaluation Question Details Widget

The Evaluation Question Details widget shows evaluation data in a crosstab table. Each row represents one completed evaluation. Questions and answers are organized across columns from left to right, allowing you to review all responses for an evaluation on a single row.

Layout Overview

The table is divided into two main areas:

  1. Left Metadata Block: Fixed information about the contact, agent, evaluator, and score.

  2. Question Columns: Evaluation questions organized by section and question number.

Left Metadata Block

The left side of the table displays fixed columns for each evaluation row:

Column Description
Master Contact Unique ID number given to the contact by the ACD. This is also called the Master Contact ID or ACD Contact ID.
Contact Number Interaction contact ID number.
Agent ID Unique ID number of the agent who was evaluated.
Agent Name Name of the agent who was evaluated.
Active Status

Status of the evaluated agent:

  • Active: The agent is in your organization.

  • Inactive: The agent has left your organization.
Team Name of the team the agent belongs to.
Team ID Unique ID number of the agent's team.
Group Group the agent belongs to.
Skill Skill for the interaction.
Evaluator Name Name of the evaluator who performed the evaluation.
Form Name Name of the evaluation form that included the question.
Form ID Unique ID number for the evaluation form that included the question.
Form Version Version of the evaluation form used for this evaluation.
Section Title Title of the section that included the question.
Evaluation Subject

Possible values include:

  • Segment: The evaluation is based on a NiCE CXone interaction.

  • Employee: The evaluation is either based on a non NiCE CXone interaction or is not interaction-related.
Evaluation Start Date Date and time (in UTC) that the evaluator began the evaluation.
Evaluation Submitted Date Date and time (in UTC) the evaluator submitted the evaluation.
Interaction Date Date and time (in UTC) the evaluated interaction took place.
Interaction Duration Duration of the evaluated interaction.
Workflow Instance ID Unique ID number of the evaluation.
Workflow Type

Type of evaluation workflow:

  • Calibration

  • Collaboration

  • Evaluation

  • Self-Assessment

Evaluation Flow Trigger

Event that started the evaluation:

  • Calibration

  • Collaborative

  • Manual

  • Quality Plan (QP)

  • Self-Assessment

Segment ID ID number of the interaction segment the evaluator was evaluating.
Evaluation Score (%) Agent's overall score for the evaluation, shown as a percentage.
Evaluation Status Status of the evaluation.
Play Click Play to play back the evaluated interaction.
Last Updated Date and time the evaluation was last modified.
Last Modified By Person who last modified the evaluation.

Question Columns

Questions are displayed to the right of the metadata block using three header levels:

  • Level 1 — Section: Displays the section name and number.

  • Level 2 — Question: Displays the question number and question title.

  • Level 3 — Sub-columns: Each question includes 14 sub-columns:

    Column Description
    Answer Full answer that the agent or evaluator gave for the question. If the answer is edited in the evaluation, it is updated in the report.
    Question Type Type of question answered. This could be yes/no, a date/time question, or a question that required a full text answer.
    Question ID Unique ID number of the specific question.
    Options All of the available answers to a multiple-choice question.
    Bank ID Unique ID number of the question bank that the question is included in.
    Critical True or False, depending on whether the question was critical. Critical questions can immediately fail an entire evaluation if they are answered incorrectly.
    Scorable True or False, depending on whether the question could be scored.
    Mandatory True or False, depending on whether the question was a mandatory question that required an answer.
    Failed Critical Question True or False, depending on whether the agent failed the critical question.
    Comments Comments from the evaluator.
    Commented By Name of the evaluator who commented.
    Commented Time Date and time the evaluator commented.
    Actual Score Score the agent received for the question.
    Max Score Maximum score the agent received for the question.

A Section Score column appears at the end of each section's question block to show the total score for that section.

On-Screen Display vs. Export

  • On-Screen Column Limit: The widget displays up to 100 columns on screen at one time. Limits are applied at section boundaries so sections are never cut off in the middle. If a report exceeds the column limit, a disclaimer notice appears in the widget. You can add/remove columns using the column selector in the widget menu.

  • Full Data Export: When you export the report to Excel or CSV, all sections, questions, and sub-columns are included regardless of on-screen display limits.

Reports with Multiple Forms

If a report contains evaluations from different forms, all evaluations appear in the same grid. Rows fill in columns only for questions on that evaluation's form. Cells for questions from other forms remain blank.